Structurally Adaptive Localized Mixtures of Experts for Non-Stationary Environments

This paper introduces a neural network capable of dynamically adapting its architecture to realize time variant non-linear input-output maps. This network has its roots in the mixture of experts framework but uses a localized model for the gating network. Modules or experts are grown or pruned depending on the complexity of the modeling problem. The structural adaptation procedure addresses the model selection problem and typically leads to much better parameter estimation. Batch mode learning equations are extended to obtain on-line update rules enabling the network to model time varying environments. Simulation results are presented throughout the paper to support the proposed techniques. 1 A mixture of experts network (from 6] 11 MSE comparison, while modeling the dual-mode Mackey-Glass data set, between the growing localized mixture of experts network and MLP 12 The number of experts in the network at any time while modeling the dual-mode Mackey-Glass 14 MSE comparison between the growing localized mixture of experts network and an MLP network while modeling the rapidly changing dual-mode Mackey-Glass data 15 The number of experts at any time in the network while modeling the rapidly changing dual-mode Mackey-Glass data 16 MSE comparison between the growing localized mixture of experts network and an MLP network while modeling the modiied Building data 17 The number of experts at any time in the growing network while modeling the modiied Building